Iterative retrieval of sparsely coded associative memory patterns

We investigate the pattern completion performance of neural auto-associative memories composed of binary threshold neurons for sparsely coded binary memory patterns. Focussing on iterative retrieval, eeective threshold control strategies are introduced. These are investigated by means of computer simulation experiments and analytical treatment. To evaluate the systems performance we consider the completion capacity C and the mean retrieval errors. The asymptotic completion capacity values for the recall of sparsely coded binary patterns in one-step retrieval is known to be ln 2=4 17:32% for binary Hebbian learning, and 1=(8 ln2) 18% for additive Hebbian learning Palm, 1988]. These values are accomplished with vanishing error probability and yet are higher than those obtained in other known neural memory models. Recent investigations on binary Hebbian learning have proved that iterative retrieval as a more reened retrieval method does not improve the asymptotic completion capacity of one-step retrieval Sommer, 1993]. In a nite size auto-associative memory we show that iterative retrieval achieves higher capacity and better error correction than one-step retrieval: 1. One-step retrieval produces high retrieval errors at optimal memory load. Iterative retrieval reduces the retrieval errors within a few iteration steps (t 5). 2. Experiments with additive Hebbian learning show that in the nite model, binary Hebbian learning exhibits much better performance. Thus the main concern of this paper is binary Hebbian learning. 3. We examine iterative retrieval in experiments with up to n = 20; 000 threshold neurons. With this system size one-step retrieval yields a completion capacity of about 16%, the second retrieval step increases this value to 17:9% and with iterative retrieval we obtain more than 19%. 4. The rst two retrieval steps in the nite system have also been treated analytically. For one-step retrieval the asymptotic capacity value is approximated from below with growing system size. In the second retrieval step (and as the experiments suggest also for iterative retrieval) the nite size behaviour is diierent. The capacity value exceeds the asymptotic value, reaches an optimum value for nite system size, and decreases to the asymptotic bound.